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Synthetic Vision (SV) may serve as a revolutionary crew/vehicle interface enabling technology to meet the 
challenges of the Next Generation Air Transportation System Equivalent Visual Operations (EVO) concept 
- that is, the ability to achieve or even improve on the safety of Visual Flight Rules (VFR) operations, 
maintain the operational tempos of VFR, and potentially retain VFR procedures independent of actual 
weather and visibility conditions. One significant challenge lies in the definition of required equipage on 
the aircraft and on the airport to enable the EVO concept objective. An experiment was conducted to 
evaluate the effects of the presence or absence of SV, the location (head-up or head-down) of this 
information during an instrument approach, and the type of airport lighting information on landing minima. 
Another key element of the testing entailed investigating the pilot’s awareness and reaction to non-normal 
events (i.e., failure conditions) that were unexpectedly introduced into the experiment. These non-normals 
are critical determinants in the underlying safety of all-weather operations. This paper presents the 
experimental results specific to pilot response to non-normal events using head-up and head-down 
synthetic vision displays. 


INTRODUCTION 

The Integrated Intelligent Flight Deck Technologies 
(IIFDT) project, under NASA’s Aviation Safety Program, 
comprises a multi-disciplinary research effort to develop flight 
deck technologies that mitigate operator-, automation-, and 
environment-induced hazards. Towards this objective, IIFDT 
is developing crew/vehicle interface technologies that reduce 
the propensity for pilot error, minimize the risks associated 
with pilot error, and proactively overcome aircraft safety 
barriers that would otherwise constrain the full realization of 
the next generation air transportation system. Part of this 
research effort involves the use of enhanced and synthetic 
vision systems and other interface modalities as enabling 
technologies to meet the challenges of an Equivalent Visual 
Operations (EVO) concept - that is, the ability to achieve or 
even improve on the safety of Visual Flight Rules (VFR) 
operations, maintain the operational tempos of VFR, and 
even, perhaps, retain VFR procedures independent of actual 
weather and visibility conditions. 

One significant challenge to the EVO concept objective is 
the definition of required equipage on the aircraft and on the 
airport. With today’s equipment and regulations, significant 
investment is required in on-board equipment for navigation, 
surveillance, and flight control and on the airport for precision 
guidance systems and approach lighting systems for “all- 
weather” landing capability. The levels of equipment 
redundancy, capability, and accuracy dramatically increase as 
landing visibility minima decrease. A vivid demonstration of 
the current complexity (and cost) to meet “weather and 
visibility-independent” capability is to look at the present 
National Air Space infrastructure. As of January 2008, there 
were 1,229 Instrument Fanding System (ILS) instrument 
approach procedures (IAPs) to Category I minima (no lower 
than 200 ft [61 m] decision height, 2400 ft [731.5 m] 
visibility) available throughout the United States, but only 143 
Category II (no lower than 100 ft [30.5 m] decision height, 


1200 ft [366 m] visibility) and 111 Category III (a decision 
height lower than 100 ft, or no decision height, or a runway 
visual range (RVR) less than 1200 ft) IAPs. 

In conjunction with the ILS sophistication, approach 
lighting systems of increasing complexity are required as 
landing visibility minima decrease. Typical for Category I 
precision approaches is the MALSR (Medium Intensity 
Approach Lighting System with Runway Alignment Indicator 
Lights) approach lighting system (ALS) and for Category 
II/III instrument approaches, the ALSF-2 (High Intensity 
Approach Fighting System with Sequenced Flashing Fights). 
An ALS provides the visual information for runway 
alignment, height perception, roll guidance, and horizontal 
references. EVO implies “Category III” operations to all 
runway thresholds. Synthetic vision (SV) displays may be one 
of the crew/vehicle interface technologies that help achieve 
the realization of EVO. 

SV is a computer-generated image of the external scene 
topography that is generated from aircraft attitude, high- 
precision navigation, and data of the terrain, obstacles, 
cultural features, and other required flight information. A 
synthetic vision system (SVS) enhances this basic 
functionality with real-time integrity to ensure the validity of 
the databases, perform obstacle detection and independent 
navigation accuracy verification, and provide traffic 
surveillance. Under NASA’s Aviation Safety 
Program/Synthetic Vision Project (1999-2006), NASA and its 
industry partners developed and deployed SVS technologies 
for commercial and business aircraft which were shown to 
provide significant improvements in terrain awareness and 
reductions for the potential of Controlled-Flight-Into-Terrain 
incidents/accidents (Arthur, Prinzel, Kramer, Bailey, and 
Parrish, 2003; Schiefele, Howland, Maris, Pschierer, 
Wipplinger, and Meuter, 2005; Schnell, Theunissen, and 
Rademaker, 2005); improvements in flight technical error to 
meet Required Navigation Performance criteria (Kramer, 



Prinzel, Bailey, and Arthur, 2003); and improvements in 
situation awareness without concomitant increases in 
workload compared to current generation cockpit technologies 
(Kramer, Arthur, Bailey, and Prinzel, 2005). It has been 
hypothesized that the use of SV technologies on head-up and 
head-down displays can provide precision approach, landing, 
and taxi guidance for “all weather” capability to all runways 
without, perhaps, requiring extensive approach lighting 
systems, ground-based precision guidance systems such as the 
ILS, or other airport infrastructure. 

An experiment was conducted to evaluate the effects of 
the presence or absence of Synthetic Vision, the location of 
this information during an instrument approach (i.e., on a 
Head-Up or Head-Down Primary Flight Display), and the type 
of airport lighting information on landing minima. A 
simulated Global Positioning System/Wide Area 
Augmentation System (GPS/WAAS) was used for approach 
guidance, instead of an ILS, and for the navigation component 
of the on-board SVS. The “operational considerations” 
evaluated under this effort included reduced visibility, 
decision altitudes, and airport approach lighting systems, for 
SVS-equipped and non-equipped aircraft. Another key 
element of the testing entailed investigating the pilot’s 
awareness and reaction to non-normal events (i.e., failure 
conditions) that were unexpectedly introduced into the 
experiment. These non-normals are critical determinants in the 
underlying safety of all-weather operations. 

The current paper documents the pilot decision-making 
process in the presence of non-normal conditions (failures and 
erroneous display information) during an approach. 

METHODOLOGY 

Subjects 

Twenty-three pilots, representing seven airlines and the 
Federal Aviation Administration (FAA) Flight Standards and 
Transport Aircraft Certification Branches, participated in the 
experiment. All but three participants had previous experience 
flying Head-Up Displays (HUDs). The subjects had an 
average of 1231 hours of HUD flying experience and an 
average of 16.8 years and 14.2 years of commercial and 
military flying experience, respectively. 

Simulator 

The experiment was conducted in the Integration Flight 
Deck (IFD) simulation facility at NASA Langley Research 
Center. The IFD emulates a Boeing B-75 7-200 aircraft and 
provides a full-mission simulation capability. The collimated 
out-the-window (OTW) scene is produced by an Evans and 
Sutherland ESIG 4530 graphics system providing 
approximately 200 degrees horizontal by 40 degrees vertical 
field-of-view at 26 pixels per degree. The synthetic terrain 
database used in the SV display concepts was created from a 
0.33 arcsec (~10 meter post-spacing) Digital Elevation Model 
(DEM) of a 53 x 57 nmi (98 x 106 km) area centered around 
the Dallas- Ft. Worth International Airport (FAA identifier: 
DFW). The DEM was draped with elevation-based generic 


imagery. The evaluation pilot (EP) occupied the left seat, as 
the Pilot Flying. The left seat included an overhead HUD 
projection unit and a head-down research display (RD). 

Head-up display. The HUD subtended approximately 26° 
horizontal by 21° vertical field of view (FOV). The FOV for 
the HUD SV (raster) image was fixed and could not be varied 
by the EP. The HUD presentation was written strictly in 
raster format from a video source (RS-343) input. The input 
consisted of a video mix of symbology and computer- 
generated scene imagery. The symbology included “haloing” 
to ensure that the symbology was highlighted against the 
scene imagery background. HUD brightness, contrast, and 
declutter controls were provided to the pilot. The SV imagery, 
when displayed, was drawn conformally. 

Head-down research display : A head-down RD was 

installed over the normal instruments on the left hand side of 
the IFD cockpit. The RD used an 18.1 in. (46 cm) diagonal 
case containing two high brightness liquid crystal displays 
(LCDs). The two separate LCD panels, each with XGA 
(1024x768) resolution, rendered a Size D (6.4 in. [41.7 cm] 
square viewable area) primary flight display, PFD (left 
display), and navigation display, ND (right display). The RD 
covered the normal Boeing 757 captain’s displays with the 
exception of the analog standby instruments (attitude direction 
indicator, airspeed, and altitude). 

The PFD and ND closely resembled current transport 
aircraft equipage. The PFD had a 50° horizontal by 50° 
vertical FOV and was only modified from typical equipage to 
include a Flight Path Marker (FPM) and guidance cue. The 
PFD FPM and guidance cue were driven by algorithms 
identical to the HUD. The ND showed the DFW Runway 18 
Right (18R) approach path, but it did not include any 
Enhanced Ground Proximity Warning System nor Traffic 
Alert and Collision Avoidance System information. 

Display Concepts 

Four display concepts (baseline and SV, either head-up or 
head-down) were evaluated while flying approaches to DFW 
Runway 18R. The head-down navigation display format was 
invariant. 

Head-up display concepts. Two HUD display concepts 
were tested, differing from each other only in the type of 
raster background (SV or none) presented. Standard HUD 
symbology enhanced with a runway outline was employed in 
both HUD concepts. Glideslope and localizer raw data 
indicators which included a deviation scale and angular 
deviation indication were provided. The path deviation 
indicators showed angular course deviation (i.e., glideslope 
and localizer-like) conditions by converting the linear path 
error data to angle errors and scaling in “dots.” For this 
experiment, one dot vertical error was equivalent to 0.35 
degrees and one dot lateral error was equivalent to 1.0 
degrees. The altitude tape included a horizontal bar 
representing the current Decision Altitude (DA), and a barber- 
pole presentation to indicate field elevation. A glideslope 
reference line was drawn at the DFW Runway 18R ILS 



descent angle of 3.0°. In Figure 1, the two HUD Concepts are 
shown - the Baseline HUD (left) and the SV HUD (right). 
During the HUD experimental runs, the “baseline” PFD was 
displayed (PFD format shown on the left side of Figure 2.). 



Figure 1. Head-Up Display (HUD) Formats - Baseline 
(left) and Synthetic Vision (right). 


Head-down display concepts. Two PFD display concepts 
were tested, differing from each other only in the type of 
background (standard sky/ground or SV) presented with the 
flight symbology. Figure 2 presents the two PFD Concepts - 
the Baseline PFD (left) and the Synthetic Vision PFD (right). 
Like the HUD, a glideslope reference line was drawn on the 
PFD at the DFW Runway 18R ILS descent angle of 3.0°. 
Unlike the HUD, the runway outline on the PFD was only 
drawn when the PFD background was synthetic vision, and 
the runway outline was drawn conformal in size to the 
selected runway and included an extended center line. For the 
head-down display evaluations, the HUD was stowed to 
preclude blocking or distortion of the pilot’s forward view of 
the outside world. 



Figure 2. Primary Flight Display (PFD) Formats - Baseline 
(left) and Synthetic Vision (right). 

Approach Lighting System Configurations 

Three different ALS configurations were used for this 
experiment. The first ALS configuration (hereinafter referred 
to as the VFR ALS) was representative of lights found at a 
VFR runway and included Runway End Identification Lights 
(REIL), precision approach path indicator lights (PAPI), 
partial threshold lights, and medium intensity runway lights 
(MIRL). The second ALS configuration (hereinafter referred 
to as the MALSR ALS) was representative of a Category 
I/Type I operations runway and included MALSR, REIL, 
PAPI, full threshold lights, and MIRL. The third ALS 
configuration (hereinafter referred to as the ALSF-2 ALS) 
was representative of a standard Category II/Type II 
operations runway and included ALSF-2, touchdown zone 


and centerline lighting, REIL, PAPI, full threshold lights, and 
high intensity runway lights (HIRL). 

Evaluation Task 

The evaluation task was a straight-in GPS W A AS -type 
approach with a 3° descent angle to Runway 18R at DFW 
airport. For each run, the approach started 5 nm (9260 m) 
from the runway threshold and the aircraft was placed one dot 
left or right of the course centerline, and one dot high or low 
of the course glide path. The weather consisted of altitude- 
based cross winds (wind direction and intensity was 
dependent on altitude), light turbulence (root-mean- square of 
2 ft/s [0.6 m/s]), and varying visibility levels (2400 ft, 1800 ft 
[549 m], or 1200 ft RVR). The wind profile started out as a 20 
knot left or right quartering head-wind and reduced to a direct 
10 knot cross-wind at field elevation. The evaluation pilot 
hand-flew the approach from the left seat with auto-throttles 
engaged at a speed of 138 knots. The run was terminated at 
full-stop or upon go-around initiation. The aircraft was 
configured to land prior to each run (landing gear down and 
flaps 30 degrees), and the aircraft was “Cleared to Land.” 
There were no other aircraft in the environment, and there was 
no ATC involvement in the task. 

Each approach had a DA of either 200 ft or 100 ft. For the 
200 ft DA approaches, the evaluation pilot was required to 
visually acquire the approach lighting system by 200 ft above 
the touchdown zone elevation (TDZE) or a go-around was 
required. If the approach lighting system was visually 
acquired by 200 ft above the TDZE, then the pilot was cleared 
to continue the approach but he/she could not descend below 
100 ft above the TDZE unless the required runway visual 
references (as defined in Federal Aviation Regulation 91.175) 
were distinctly visible and identifiable by natural vision. For 
the 100 ft DA approaches, the evaluation pilot had to visually 
acquire the required runway references (e.g., threshold lights) 
by 100 ft above the TDZE in order to land. Otherwise, a 
missed approach was executed. 

The evaluation pilots were instructed to fly the aircraft as 
if there were passengers aboard, fly the center of the approach 
path, and land as close as possible to the centerline and 
touchdown zone (1000 ft [305 m] from the runway threshold). 
After landing, they were to capture the center line and come to 
a complete stop on the runway. They were also instructed to 
initiate a go-around if the landing was not safe or there were 
any safety concerns during the approach. 

Experiment Matrix 

Nominally, ten training runs and fifty-three experimental 
runs were completed by the EP. The primary experiment 
matrix consisted of combinations (but not a full-factorial) of 
Display Concept (Baseline HDD, SV HDD, Baseline HUD, or 
SV HUD), ALS (VFR, MALSR, or ALSF-2), runway 
visibility range (1200 ft, 1800 ft, 2400 ft), and DA (100 or 
200 ft) as shown in Table 1. 

A significant component of the test, in addition to the 
nominal runs, was the investigation of the ability of the EP to 




recognize and properly handle non-normal events. These non- 
normal runs were injected into the test unbeknownst to the 
EPs. Seven non-normal runs were flown by each EP. The non- 
normal runs were four navigation system error scenarios, two 
altimetry failure scenarios, and one SV obstacle placement 
error scenario. Each of the non-normal runs used the MALSR 
ALS with a 100 ft DA and 2400 ft RVR. The number of non- 
normal scenarios was designed to avoid expectancy on the 
part of the flight crew (Foyle and Hooey, 2003). The 
navigation system error scenarios purposefully introduced 
either a lateral navigation solution error (40 m right) or an 
altitude navigation solution error (50 ft [15 m] low) with 
respect to the real runway. This error resulted in the synthetic 
vision terrain and guidance cue being misaligned from the 
ALS and actual runway environment. The altimetry failure 
scenario emulated a failure inhibiting the aural callouts and 
the display of radar altitude information and flare cue 
symbology to the pilot. The intent of this scenario was to 
assess the impact of the aural callouts and radar altitude 
display on the pilot’s ability to be cognizant of the DA, make 
the appropriate pilot callouts (e.g., “lights”, “landing” or “go- 
around”), and perform a safe approach and landing. In the SV 
obstacle placement error scenario, a water tower which was 
located 2 miles (3.2 km) from the runway threshold in the 
OTW view (present on every run), had its SV icon 
(representing the obstacle) erroneously placed only 1 mile (1.6 
km) from the runway threshold. This scenario results in a 
mismatch of the SV imagery with the OTW scene for the 
water tower obstacle and presents a prominent SV-depicted 
obstacle along the aircraft’s flight path. 


Table 1 : Primary Experiment Matrix 
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2400 RVR 100 ft 

V i 
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200 ft 
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Note: the checked cases were evaluated for all four display concepts 


RESULTS 

Nominal run objective and subjective data analyses from 
the present study were previously reported (Kramer, Williams, 
and Bailey, 2008; Kramer, Williams, Bailey, and Glaab, 2007) 
and are summarized below. 

• Addition of SV imagery on the head-up or head-down 
displays did not have an adverse affect on approach and 
landing performance, and was, in fact, the pilots 
preference due to the significant increases it provided in 
situation and spatial awareness (without any workload 
penalties). 

• SV HUD was rated as providing the greatest situation 
awareness and lowest workload among the 4 concepts 
tested. 


• For the visual portion of the flight maneuver (i.e., 
landing), RVR appeared to be the most significant 
influence on touchdown performance. 

• Type of ALS did not appear to influence touchdown 
performance or approach path maintenance. 

The next sections document the pilot’s decision-making 
process when confronted with seven non-normal events (4 
navigation system errors, 2 altimetry failures, and 1 SV 
obstacle placement error) while using the MALSR ALS with a 
100 ft DA and 2400 ft RVR. Of particular relevance to this 
paper and the subsequent discussion of the EPs reaction to 
non-normal events is that all nominal approaches (92 total 
runs) flown in these same flight conditions (MALSR ALS, 
100 ft DA, 2400 ft RVR) resulted in a landing (i.e., no missed 
approaches) (see Kramer et al., 2008). 

Lateral Navigation Error 

Each pilot experienced two runs with a lateral navigation 
position error (40 m right on breakout), each occurring with 
the same display location (head-up or head-down), but with 
one evaluation using the baseline information content and the 
other using the SV information content. For all pilots, 43 
landings were made out of 46 approaches (93%) despite the 
lateral position error. Of the three missed approaches flown, a 
go-around occurred for one Baseline HDD run, one SV HDD 
run, and one SV HUD run. With the Baseline HDD run, the 
pilot called “go-around” when he didn’t think he had enough 
time to recover from the error. This same pilot also went 
around with the SV HDD run. On the SV HUD missed 
approach condition, the pilot called “go-around” and then 
post-run, commented that there must have been a lateral shift. 
To test the influence of a substantial lateral navigation error 
on touchdown performance, an ANOVA was performed on 
touchdown lateral position from centerline. The effects of run 
type (nominal, 40 m offset), F(l,21)=0.433, p=0.52, and 
display concept (SV HUD, Baseline HUD, SV HDD, Baseline 
HDD), F(3,84)=0.445, p=0.72, were not found to be 
significant for this measure. Run type means were 8 ft (2.4 m) 
for the nominal runs and 7 ft (2.1m) for the 40 meter offset 
runs. Display type means were 9 ft (2.7 m) for the Baseline 
HDD and 8 ft for the other three display concepts. Visual 
inspection of the landing data revealed that all touchdowns 
occurred on Runway 18R and ended safely with a full-stop. 
One could infer that the presence or absence of a 40 meter 
error did not affect the touchdown performance. 

Altitude Navigation Error 

Every pilot saw two altitude navigation error runs (50 ft 
low on breakout) during data collection, each happening at the 
same display location (head-up or head-down) with one using 
the baseline information content and the other using the SV 
information content. 44 of the 46 (96%) altitude navigation 
error runs ended in a landing. A go-around occurred for one 
SV HDD run and one SV HUD run. On the SV HDD run, the 
pilot correctly followed pilot procedures and performed a go- 
around as he had the approach lights but not the runway 
environment through natural vision at 100 ft TDZE. Hence, 




this go-around was not due to the altitude navigation error. On 
the SV HUD run, the pilot called go-around once he broke out 
and saw that he was low on the approach. The presence of a 
50 ft low altitude navigation solution error appears to not have 
adversely affected the missed approach rate (2%, or 1 out of 
46, with altitude error versus 0% with nominal runs) but it did 
affect touchdown performance. Visual inspection of the 
landing data revealed that two of the altitude navigation error 
landings occurred before reaching the Runway 1 8R threshold 
- one with the Baseline HUD (landed 39 ft [11.9 m] aft of 
threshold) and one with the SV HUD (landed 102 ft [31.1 m] 
aft of threshold). To test the influence of a significant altitude 
navigation error on touchdown performance, an ANOVA was 
performed on the touchdown longitudinal distance from aim 
point for the 42 landings that occurred on Runway 18R. 
There were no significant run type effects, F(l,23)=0.434, 
p=0.52, or display type effects, F(3,74)=0.717, p=0.55, for 
this measure. 

Altimetry Failure 

Each pilot saw two altimetry failure runs during data 
collection, each happening at the same display location (head- 
up or head-down) with one using the baseline information 
content and the other using the SV information content. Of the 
altimetry failure runs 45 of 46 ended in a landing. The one 
altimetry failure run that ended in a go-around was due to the 
pilot noticing that the Baseline HDD didn’t have the radar 
altitude information and so he decided to go-around. The pilot 
commented that he had the lights required for landing. The 
presence of an altimetry failure did not appear to adversely 
affect the missed approach rate (2% with altimetry failure 
versus 0% with nominal runs). ANOVAs were performed on 
touchdown lateral distance from centerline and longitudinal 
distance from aim point. There were no significant run type 
effects, F(l,21)=0.005, p=0.94, or display type effects, 
F(3,84)=0.633, p=0.60, for lateral distance from centerline. 
Similarly, no significant run type effects, F(l,22)=1.068, 
p=0.31, or display type effects, F(3,86)=0.794, p=0.50, were 
found for longitudinal distance from aim point. All landings 
occurred on Runway 18R and safely ended in a full-stop. 

Obstacle Placement Error in SV Database 

Each pilot saw one SV obstacle placement error run 
during data collection. Twelve of the pilots encountered the 
SV obstacle placement error on the SV HDD concept and 1 1 
of the pilots encountered the error on the SV HUD concept. 
During this run, 4 out of the 12 pilots (33%) went around 
when using the SV HDD and 9 out of the 11 pilots (82%) 
went around when using the SV HUD. During an approach, 
the SV obstacle icon misplacement was recognizable sooner 
on the HUD (-1700 ft [518 m] mean sea level [MSL]) than on 
the HDD (-1200 ft [366 m] MSL). The symbology and 
imagery on the HUD is conformal with the out-the-window 
view; while, the symbology and imagery on the HDD is 
minimized. In addition, the obstacle icon on the HDD was 
somewhat occluded by the magenta guidance cue and runway 
outline with extended centerline. For one of the HUD 
landings, post-run the pilot commented that he knew that the 


obstacle was in the approach path but he also knew that he 
was on glide path and so he just assumed something was 
being drawn incorrectly on the display. For the other HUD 
landing, the pilot commented post-run that at 320 ft (98 m) 
above ground level he noticed what he thought was an 
extended runway centerline. He didn’t realize it was the 
obstacle icon that he was viewing as he had seen the real 
water tower OTW prior to this point in the approach. For one 
of the HDD landings, the pilot reported the tower OTW 
during the run and post-run commented that the icon was not 
drawn correctly with regard to the tower OTW. During one 
approach, a pilot commented on misplacement of the obstacle 
icon, leveled off momentarily, and then decided to continue 
the landing. Two pilots commented post-run that the water 
tower symbol seemed to “pop up” all of a sudden on the 
display and that it happened too suddenly to react to it. Four 
of the pilots made no comment at all on the SV obstacle 
placement error on the HDD. 

DISCUSSION 

Pilot decision-making and reaction to non-normal events 
such as a navigation solution error or an altimetry failure was 
not adversely affected by the addition of synthetic vision (SV) 
imagery on either the head-up or head-down displays. 
However, display minification negatively impacted pilot 
identification of an SV obstacle placement error within the 
imagery and subsequent recognition and reaction to it. Future 
research experiments should examine the feasibility of using 
vision technologies like synthetic and enhanced vision (e.g., 
for SV obstacle integrity checking and aim point verification) 
to provide for an all weather (visibility) landing capability 
without the need for a visual approach segment. 
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